Aminet 2

home *** CD-ROM | disk | FTP | other *** search

/ Aminet 2 / Aminet AMIGA CDROM (1994)(Walnut Creek)[Feb 1994][W.O. 44790-1].iso / Aminet / dev / misc / ppipc.lzh / Docs / IPC.Doc < prev next >

Wrap

Text File | 1989-12-01 | 34.6 KB | 664 lines

An Inter-Process Communication Standard for the Amiga ===================================================== -- November 1989 -- This is initial documentation for the IPC standard developed largely in the course of some furious traffic over usenet during 1988. The core of the standard is now stable (the IPCPort mechanism and the IPCMessage structure), but there is work still to be done on the details, on developing message IDs for specific functions, and on publishing them. Also elements such as the "Port Broker" need more development. [There are no doubt still many gaps in this document. I intend to fill them in as time goes on (and permits!), but please bring any egregious omissions to my attention.] Why IPC? ======== [Since the following paragraphs were written, ARexx has come into fairly widespread use, and is obviously an attractive way of tackling some kinds of IPC. However, the mechanism to be discussed here has a rather different focus, and can be used in situations where, for example, speed limits would rule out ARexx. More will be said on this later.] Up to now, even though the Amiga is the first really totally multitasking personal computer, programs for it have been written much as for any of the older one-job-at-a-time machines (though they often take good advantage of the windowing environment): they run as single processes. A few programs spawn child tasks to perform some of their functions, but they are still monolithic in their design. There is an alternative. A ... let's call it a "Job environment"... could be serviced by a number of more specialized processes, communicating with each other. Depending on the needs of that environment, these processes could be very specialized -- a note bender in a MIDI system, say -- or more general, like a text editor that called up things like a spell checker or output formatter when needed. Such a system would give the user much more choice in configuring it to his/her preferences. (Not forgetting though that a default configuration would have to be as easy to use as any well designed program!) The modules in an environment wouldn't all have to come from one company, as the interface to invoke a module's function would be published with the module. You could add special purpose functions to a paint program say, just by starting up another process that would recognize, or be recognized by, the paint program. There are a lot of possible applications of this principle, with widely varying requirements. Here is a very incomplete list of a few that come to mind: * Communications -- connect terminal emulators, protocol handlers [Kermit, XModem etc.], auto-answer modules, script processors etc. to suit the user's environment. [Note the wide range of parameters such as processing speed demanded by different modules: protocol handlers will have to operate character by character at baud rates, while script servers and auto-answer can be much more leisurely.] * MIDI Modules -- a lot of possibilities here for a fast IPC protocol; anything you might want to do to a MIDI channel could be a "plug-in" module. * Data Acquisition and Control -- there are untapped opportunities for a multitasking machine in the laboratory; being able to "plug" modules together like you can with lab equipment would be a great attraction. * Desktop Publishing -- put together a comprehensive and competent system from the pieces YOU prefer. * Desktop Video and Presentation -- lots of enticing possibilities here; the real-time nature of presentation gives lots of opportunities for fast IPC. (One might even think of coupling several machines together...!) * Service Programs -- to do common jobs for other modules; small ones -- print format servers, pattern matchers, directory searchers etc. -- can take a lot of the load off other modules that would otherwise have to include code for those things; larger ones could handle jobs like spell- checking for any program that wanted it. * A Multitasking H*perC*rd -- a somewhat vaguely specified, but highly enticing, goal; dream your own scenario... This should give some flavor of the ways in which I think a general "Cooperating Processes Paradigm" would be a winner. You can see that a wide range of schemes could fall under the term "IPC". It would be nice if they could all be handled by one underlying protocol. There are a number of possible candidates. Processes can pass information between each other via files and pipes, for example. This has the great advantage that it is already built into the system, but its disadvantage is the heavy overhead involved in file-type I/O; passing around large blocks (like bit maps) is probably ruled out. Also, simple serial stream, unstructured, communication isn't good enough for a system that is going to pass around many different types of information. The receiver of information has to know exactly what it is getting, and the sender must know exactly how to tell the receiver about it. This means some kind of standard structure for the data that can be understood by all processes. IFF is such a standard, but again unfortunately it is too complex and serial-oriented for very fast communication. Its principles make a good starting point though. There is an obvious candidate built into the Amiga Exec: Messages and Ports. These are actually just about what we want, but there are a couple of problems with them as they stand. First, although ports work perfectly when child tasks spawned and managed by a single master process use them to talk to each other, there is no protection against accidents that are likely to occur when independent processes communicate. The problem is that, as ports are accessed by a pointer, there is no way of being sure that the port pointed to still exists when you want it, unless you lock out all other processes before you get the pointer, and don't unlock them again until you've used it. This can get VERY cumbersome and inefficient if you want to send a lot of messages to a port, as you have to do the Forbid/Find pointer/Use pointer/ Permit sequence for every message. Second, there is the problem of a process knowing exactly what is in the message it has received. As we said, there must be a Standard Structure for the data in the message so that it becomes easy for different authors to write programs that can understand each other. Hence the IPC Standard really breaks down into two parts: IPCPorts and IPCMessages. These are essentially independent of each other, in that the Ports have no knowledge or concern about the message structure (aside from the necessity for the standard Exec Message structure at its heart), and conversely the Messages are not concerned with the Port procedures. It should be pointed out that this is a "Foundation Level" protocol. It defines how ports should be managed, and the common structure of messages, but says nothing more about how they are to be used. There are many higher level considerations, such as how the processes should appear to the user, and how he is to control them, that need to be worked out. There are a lot of possibilities here: control panels, patch panels, "drop-in" menus...; some applications will need added capabilities such as command languages and scripts; others would be slowed unacceptably by such overhead. But with luck this standard will be general enough to cover the range. There are other schemes, actual and potential, that have to be considered in relation to this one. In general there is probably no need for them to be competitors. One such is AREXX, which has already been successful in applications like integrating editors with text formatters; because it is basically an interpreted script language, it doesn't seem so suitable for some of the other potential applications mentioned above. On the other hand, because it is a script language, it is more immediately accessible to the user than this IPC protocol, which only addresses the program level. There seems no reason though why an interface between AREXX and this standard could not be written; only a restricted set of IPCMessages could be handled, but probably enough to give great flexibility. One other thing this standard has in its favor. It is Public Domain... I've put copyright marks in the code sources, but this is solely to discourage incompatible modifications. Both the protocol and the library may be used in any program or product, for profit or otherwise, without fee. The main thing I ask is that you treat any new message IDs you assign (see the next section) as similarly public, and try to keep as much standardization as possible in these. Before defining any new IDs, see if the ones described in the "Standard IDs" document will do, and if not, pass the ones you create around (and specifically back to me! I guess I have to be the main repository at the moment). When developing a set of IDs, try to think beyond the needs of the moment (this ain't always easy!). If you really find you need to modify the library itself, please communicate with me if possible first. The IPC Protocol ================ As was said, this IPC mechanism is a general way for two (or more) processes to communicate quickly and conveniently, using "IPCMessages". The channels through which these messages are passed are "IPCPorts". For clarity we talk about the original source of a message as a "Client", and the receiver that processes it as a "Server". These terms are relative: a process may be a Client for one channel and a Server on another. Often a process will spend a lot of its time processing the messages it receives on one channel by sending other requests out on other channels; I've taken to calling such a process a "Manager". There is one particular Server that needs brief mention here, even though it is largely independent of the protocol itself -- the "Port Broker". A Client may well request an IPCPort that isn't yet attached to a Server; it would be nice if this situation could be handled automatically, and this is in fact the function of the Port Broker. This is a process that will be resident continuously; when a port is requested (by the "LoadIPCPort" call) that does not yet have a server, a message is sent to the broker, which looks up that port name in a list it has been provided with, and finds (and executes) the command needed to load a server for that port. As with other Servers, there should eventually be a choice of Brokers for the user; there are a lot of functions in addition to the basic one that they could provide: more intelligent management of Servers (and possibly Clients) is one. The "Broker" directory in this package contains a simple server and demo, and a more extensive discussion. The messages themselves can "contain" any information needed for the activity they are concerned with. The quotation marks are because the information most often won't be in the message block itself but rather in a separate block pointed to by an item in the message. (And remember that a message doesn't "move" when it is passed: it is simply put on a list in the IPCPort -- i.e. a couple of pointers are changed -- for the server to pick up.) A message is "self-identifying". A given IPCPort can handle many different types of IPCMessages, and the Server processing them will be able to determine immediately exactly what each contains. It may alternatively find that it doesn't recognize the message, or some parts of it -- perhaps because the Client was assuming a more capable Server -- and will have to send it back with a rejection slip. You get a number of levels of selection. First, and most trivial, is the name of the IPCPort: a Client, before it can send messages to a port, will (normally) have to know its name; this can be any ASCII string, but we can expect that alternative Servers that provide equivalent functions will use ports with the same name. This provides a first line of defence against wildly inappropriate messages, but it obviously is not nearly enough. As we said, the Server must be able to determine the exact type from the message itself. It does this from the "Message IDs", the "Item IDs", and to some extent from flags within the message. Each message type has a specific 32-bit ID which exactly delimits its contents. The ID normally comprises four upper-case ASCII characters (exactly like an IFF ID word). Some IDs are reserved to mean exactly the same thing across all servers. Others will be meaningful only to particular Server functions; these should be chosen if possible not to clash with IDs used by others, but there should be little problem in practice if a clash does occur because they will be received by totally distinct Ports and Servers. The internal structure of a message is totally consistent. Within the message will be a number of "IPCItems", the exact count being specified in the header. Each item has its own ID -- of exactly the same form as the message ID -- and always has the same (sixteen byte) structure, the most important element of which is a pointer to the data (if any). A Server can quickly determine from scanning the various IDs what the message means, and can take the needed action. If permitted by the context, it can write new data to some of the blocks in the message, or even fill empty slots in the message with new data (pointers). It then replies the message to the original Client, and the transaction is complete. IPCPorts -- and IPCMessages -- are managed by a resident shared library (IPC.lib), which must naturally be available in the LIBS: directory. Depending on the compiler you are using, you may need to include a small linkage module in your executable code to access the library functions (see the README file in the parent directory of this package). All declarations and definitions needed for IPC are in IPC.h (suitable for all C compilers). For more information on the ways to incorporate IPC in the programs you write, please refer to the README file. A number of examples are also provided. The following sections go into details of, first, the IPCPort mechanism, and then IPCMessages. +++++++++++ IPCPorts ======== An IPCPort is a rendezvous point between processes where messages can be dropped off and picked up. It is identical in concept to the standard Amiga Exec MsgPort (see the Rom Kernel Manual), but is managed so that messages cannot be sent to a non-existent port (if the processes obey the rules!). Any number of "Client" processes can be sending messages to a port at one time, but there can be only one "Server" on a port. If a port doesn't have a Server -- either running or in the process of being loaded --, it is "Closed", and won't accept messages. Unlike an Exec MsgPort, an IPCPort doesn't "belong" to any particular process, and a program must never create one directly or assign memory in its own space for one: a port will be created by the first process that needs it, and will remain available until all references to it have been cleared; it may be finally deleted by the last user long after the first has gone away. Either a Server or Client may make the first reference. Each port is identified by a unique name (although unnamed anonymous ports are also possible, as we'll see later); the name may be any appropriate null-terminated byte string. A port is located by an exact match to the string: the case of the characters is respected. The author of a server will publish the names of the ports it services, so that others can write clients to access it; if two programs offer alternatives for the same service, they should use the same port name, so that the user may plug one of them in without changing the client. Clients and IPCPorts ==================== When a Client process first needs a port of a particular name, it must get a pointer to that port. The pointer will then remain valid until the Client specifically drops its access request. Each request made MUST be paired eventually with a drop, for correct port management. There are three possible ways of getting a port pointer. Each of these procedures takes a names string as argument and returns the pointer if possible. They differ in the actions they take to ensure the status of the port. GetIPCPort(name) always returns a valid pointer (unless memory is full, or some other disaster strikes). The port is created if it doesn't exist. No indication is given as to whether there is a current server. FindIPCPort(name) only returns a pointer if the port already exists and has a server; otherwise it returns NULL. LoadIPCPort(name) does a GetIPCPort, but if the port has no server it calls on a "Port Broker" process (only an early version of such is provided in the system at this stage) to load one; if the broker can't do this (or the broker doesn't exist) the port will be dropped again, and NULL will be returned. If a valid pointer is returned, you can assume that a server exists or is being loaded. LoadIPCPort does not wait for the server to be loaded, but the "Loading" flag will be set in the port so that it will accept messages for the server to handle when it arrives. [The "Port Broker" Concept is intended as a core part of the IPC system, but has at this writing only been implemented in a simple form. This is in its own directory in this suite, where you'll find more discussion. I hope that others will develop Brokers of their own; this is, after all, the philosophy of IPC!] Each successful call to one of these procedures increments the "Use Count" of the port. To reduce the use count again you must end the Client's access to the port by: DropIPCPort(port) where 'port' is the pointer returned by any of the above calls. The use count is reduced by one: if it goes to zero, the port is deleted. A Client must not exit without dropping all the ports it has acquired (and of course must not drop them more times than it has acquired them!). A Client sends messages to an IPCPort by a call exactly analogous to Exec's PutMsg(), but that Exec call has no way of detecting the presence or absence of a Server, so it should not be used. Instead use: PutIPCMsg(port,message) where 'port' is a valid pointer, and 'message' is a pointer to an IPCMessage. It returns TRUE if the message was successfully queued on the port; if the port has no server either present or loading, it will not send the message and will return FALSE. (If there are good reasons for bypassing the protocol -- as when an IPCPort is used as a reply port, see below -- a PutMsg (or ReplyMsg) call can be used, PROVIDED that you know that there is a Server, or will eventually be one; if there is no Server, one that attaches to the port later will pick up the message; on the other hand, if no Server ever finds the message, it (and the memory it occupies) will be gone forever.) Unless there are specific reasons otherwise, a message sent to a Server will eventually be replied, and the Client must handle this also by supplying a Reply Port. The reply path does not need the IPCPort protocol, because a Client MUST keep the reply port available until it has received back ALL the replies it is expecting, so normally this can be a standard Exec MsgPort. There is no reason though why it should not be an IPCPort as well if you prefer -- even one used for other communications -- as long as you bear in mind that replies will be sent to it whether or not it has a server assigned. Servers and IPCPorts ==================== The Server uses a complementary set of procedures to a Client to manage a port. There is only one way it can acquire a port: ServeIPCPort(name) makes this process the server for the named port if possible and returns a valid pointer to the port. The port is created if it doesn't exist. If a server already exists for the port, the call will return NULL. (If successful, it also increments the use count.) Terminating service on the other hand is normally a two-step process. First the server must shut the port against further incoming messages, then handle and reply to any messages still queued on the port, and finally free up the port for eventual reclamation or for another server to claim it. ShutIPCPort(port) just marks the port as "Shut". The server remains attached to the port and can receive signals from it (for example if the number of clients changes, see below), but PutIPCMsg calls will be blocked. LeaveIPCPort(port) removes this process from association with the port. Also does a DropIPCPort to decrement the use count and delete if appropriate. Between the ServeIPCPort and the ShutIPCPort, the server must be prepared to accept messages on that port. In most cases it will probably spend the major portion of its time waiting on that port -- or possibly several ports -- for a message to arrive. All this area of the server's operation is handled by standard Exec calls: WaitPort() or Wait(), GetMsg(), and ReplyMsg() (see the Rom Kernel Manual). If it only has one port to wait on, it can use: WaitPort(port) which suspends the process until a message arrives at that port. This is liable to be inadequate though, because it will only be woken up by messages arriving on that particular port. It is NOT awakened by other signals to the process, even if they use the same signal bit (see the RKM for Task Signals). Thus you are more likely to want to use: Wait(sigbits) where 'sigbits' is a 32-bit mask of all the signals you want to be awakened on. It returns the signals that were actually set, so the process can determine if it needs to take unusual action. This means, though, that you have to create the sigbits mask from the signal bit number contained in the port itself (and other bits that you want to respond to). There is no specific call to get this (though there should probably be at least a macro), but if the server is just using IPC.h as a header (without IPCPorts.h -- which should only be needed for IPC system management programs) IPCPorts are equated to MsgPorts, so you can get the bit by: sigbit = 1L<<port->mp_SigBit; Note that this bit is also used by the "Notify" feature (below) to awaken the server if the number of clients changes. No message is associated with this signal, but all the server has to do is check the number of clients (again, below) each time it is awakened. To process a received message, the server simply uses GetMsg(), though it is preferable to cast the returned value to the correct type: msg = (struct IPCMessage *)Getmsg(port); When done with the message, it should use ReplyMsg() in the normal way (again using a cast, preferably): ReplyMsg((struct Message *)msg); Checking IPCPort Status ======================= Either Client or Server can get information about the current state of an IPCPort with the CheckIPCPort() call. The Server alone can also set the state of certain flags in the port (only one is currently defined). CheckIPCPort(port,flags) as long as the high bit of 'flags' (a 16-bit value!) is not set, returns the number of users (including the server) currently aware of the port; if the high bit is set (0x8000 -- this should be a defined identifier... an omission in the current IPC.h, sorry), the call instead returns the current flags set in the port. When called by the current Server ONLY, the value in the low 8-bits of the 'flags' argument will be set into the port's Flags slot; system flags in the upper 8-bits of the slot are not affected. Although a Client may use this call, it will probably have no need to, unless it wants to check for the presence of a Server without making a PutIPCMsg call. The Server, however, can use the call to keep tabs on how many clients actually require its services, and optionally terminate if there are none. To do this it will want to be notified if the number of clients changes; this can be done by setting the IPP_NOTIFY flag in the port: while that is set, any call that acquires or drops a port will send a signal to the Server process (using the signal bit defined for that message port); no message is actually sent, but the Server should call CheckIPCPort each time it is woken up. Anonymous IPCPorts ================== It is possible to create IPCPorts that are unnamed, but these obviously cannot be acquired independently by other processes without a name to access them by. The associated pointers must be passed to other processes via some standard message pathway (see the companion document on Standard IDs). They still should obey the IPC rules, though, so another procedure is provided for acquiring a port passed as a pointer: UseIPCPort(port) registers another user for that port (i.e. increments the use count). (Note that you are not restricted to passing pointers to unnamed ports, and UseIPCPort can be applied to ANY IPCPort when a process has its pointer rather than its name.) When the user process is done, it issues DropIPCPort in the usual way. You create an anonymous IPCPort by: ServeIPCPort(NULL) which will return a pointer to a NEW port for each call, and set the caller as the Server. (It will only fail if there is no memory or some such fatal problem.) For special cases -- e.g. ReplyPorts -- where you know PutIPCMsg will never be used, you could instead use GetIPCPort(NULL), but there is NO way of registering a server on an anonymous port except at creation time. +++++++++++ IPCMessages =========== The format of IPCMessages is intended to be very flexible, yet specific enough that a Server can immediately determine the function and content of a message it receives (or reject it if it does not recognize it). The protocol has two levels where identification is specified. There is a fixed structure Message header (basically an Exec Message structure with added information) which includes a Message ID, followed by an arbitrary number of "Items", each again of fixed structure with its own Item ID. Each item in turn usually points to a block of memory where the actual data for the item is stored. Instead of a pointer, a single 32-bit value can be stored in the item itself. In special cases, the data pointed to may be more complex than a single data block (a list, say) but then the restrictions on what can be done with the message increase markedly. As long as all the items in a message are simple data blocks (or single words), it can be passed between servers that do not have to understand the contents of those blocks: deletion of the message, for example can be done by any server. Flags in the message and each item indicate whether the data is private to the originating process or can be transferred to the receiver, whether it is suitable to send out on a network, and so on. An ID -- Message or Item -- is a 32-bit longword, normally a four-character ASCII code. All "Published" IDs should be of this form, but cooperating processes could perhaps use non-ASCII numeric values internally. If the first (high) byte is zero, the ID is a private one. Codes of less than four characters may of course be used, but they preferably should be padded on the right with blanks (rather than nulls). Case is important naturally; by convention (and analogy to IFF) upper case is preferred, but there is no firm requirement for this; lower case should probably be used to extend on meanings that were originally upper case. For details on IPCMessage structure, and the various flags currently defined, please refer to the IPC.h header file (and see the following section). A full description of the flags is in the accompanying document "Standard Flags". A list of the currently reserved IDs, and others of more specific utility, is in "Standard IDs". For the ways these IDs (or some of them) have been used, look at the demo example sources in this package. When you develop your own Servers, you will of course immediately need to assign your own IDs; please do it with future expansion in mind, and publish them on usenet or elsewhere as soon as possible for comment. Message Structure ================= This section is just a restatement (and maybe clarification) of the information in IPC.h. Please look at that file also. Each message has a header which is a standard message structure, followed by an ID field (32-bits), a 32-bit Flags field, and an Item Count (16-bits). This is immediately followed by that number of items, each of which has the following structure: Item ID (32-bits), Flags (32-bits), Size of data block (32-bits), and Pointer to data block (also of course 32-bits). This may in turn be followed by a data area containing the data for some or all of the items in a message; putting data here may or may not be convenient, depending on the nature of the data. The mn_Length field in the standard message structure indicates the complete length of the message (remember it is only 16-bits, so the maximum size of a message containing in-line data is 64K). As already remarked, the "Pointer" field of an item usually points to a single block of data, but it may represent other things as well. It may be a single value, or a non-standard item (a BPTR for example). In these cases, the Size of the item should be set to ZERO. As long as the Server only has to treat the item as a value or a handle, this is all that needs to be done. In other situations it may be a true pointer to data which the Server must deal with, but the data referenced may NOT be a simple data block (created by AllocMem); it could be a list structure, for instance, or it could be itself a structure which contains pointers to other data. If this is so, the IPC_NONSTANDARD flag must also be set, so that "ignorant" servers won't do anything silly. The contents of the Size field here will depend on the function of the item, but most often it will probably be zero. Managing IPCMessages ==================== Two system procedures are provided for convenient management of memory for IPCMessages (much preferable to the program doing it all for itself). CreateIPCMsg(items, extra, replyport) creates a new message block for that number of 'items' (0..n), with 'extra' bytes of memory after the items for data storage (you will have to get its offset as the location of the "item" following the last actual item); the 'replyport' pointer is inserted in the standard message structure. A pointer to the created message (or NULL if there is no space) is returned. DeleteIPCMsg(msg) deletes the memory block created by CreateIPCMsg. It does NOT attempt to handle memory occupied by data outside the message and pointed to by its items. +++++++++++ Other Library Functions ======================= Two other functions have been added to the library for user convenience: MakeIPCId(string) returns a longword ID value constructed from the characters in the supplied string. The string should normally be four uppercase ASCII characters; if you supply fewer they will be high-bit ("left") justified and padded with NULLS (but see notes on preferred IDs above); more will just be truncated. This function will be useful for programs whose set of IDs is not fixed at compile time (they might read them perhaps from a setup file). FindIPCItem(msg, ID, firstitem) returns a pointer to the first item found in 'msg' that matches 'ID' (NULL if none is found). If 'firstitem' is not NULL, it MUST be a valid pointer to an item in the message, and the search will begin from (and include) that item; if it is NULL, the search will begin from the first item in the message. If you are doing successive searches for the same ID, remember to increment the 'firstitem' pointer before the next call -- otherwise the same item will be returned repeatedly! +++++++++++ Pete Goodeve November 1989